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Abstract: We study the set of boundary orbit accumulation 
points of the automorphism group action on a bounded domain in 
C n . Topological and geometric properties of this set are derived. 



Introduction 

A domain Q in C n is a connected, open set. An automorphism of Q is a 
biholomorphic self-map. The collection of automorphisms forms a group 
under the binary operation of composition of mappings. The topology on 
this group is uniform convergence on compact sets, or the compact-open 
topology. We denote the automorphism group by Aut(fi). 

Although domains with transitive automorphism group are of some in- 
terest, they are relatively rare (see [HEL]). A geometrically more natural 
condition to consider, and one that gives rise to a more robust and broader 
class of domains, is that of having non-compact automorphism group. Clearly 
a domain has non-compact automorphism group if there are automorphisms 
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{(fj} which have no subsequence that converges to an automorphism. The 
following proposition of Henri Cartan is of particular utility in the study of 
these domains: 

Proposition 0.1 Let f26C n be a bounded domain. Then Q has non-compact 
automorphism group if and only if there are a point X G Q, a point P G dQ, 
and automorphisms ipj of Q such that (pj(X) — > P as j — > oo. 

We refer the reader to [NAR] for discussion and proof of Cartan's result. 

A point P in dfl is called a boundary orbit accumulation point if there is 
anlGfl and automorphisms tpj of Q such that lim :; _ i . 00 fj(X) = P. 

The inspiration for the present paper comes from [ISK1]. In that paper, 
we considered the set S(Q) of all boundary orbit accumulation points of a 
smoothly bounded, finite type domain Q in C n (see [KRAI] for the concept of 
finite type). In particular, we showed that, if the set S contains at least three 
points, then it is perfect. Also, if S is not connected, then it has uncountably 
many connected components. Finally, the set S is generically contained in 
the set where the Levi form has minimal rank and where the type and the 
multiplicity functions are maximal. 

Our goal in this paper is to extend and generalize these results. We also 
provide examples to show that some of these statements are sharp. 



1 Some Examples 

We begin with some examples. These will all be smooth deformations of the 
unit ball in C 2 . 

EXAMPLE 1.1 There is a bounded domain f2 in C 2 with Lipschitz bound- 
ary with the property that S(Q) has just two points. 

To see this, let B be the unit ball in C 2 . For j G Z, let 



Zi-1/5 y/l-(l/5) 2 Z2 



1-(1/5)V l-(l/5)zi 



and let ip^ denote the composition of tp with itself j times (and of course 
ipO = ((p- 1 )!-?! when j < and (p° = id). 

Then each pp is an automorphism of B. Furthermore, 

lim (p 3 (z 1 ,z 2 ) = (-1,0) , 
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uniformly on compact subsets of (zi, z 2 ) in B. And 



lim (p j (z 1 ,z 2 ) = (1,0) , 

uniformly on compact subsets of (-21,-22) in B. 
Now define 

n(z z) - J (1/100 -N 2 ) 2 - (1/100 -M 2 ) 2 if 1(^1,^)1 < 1/10 

«^1^2j-| q if 1(^,^)1 > 1/10. 

We set 

Q' = {( Zl ,z 2 ) G C 2 : (|zi| 2 + |z 2 | 2 - 1) + a(*i -i,-2 2 ) < 0}. 
Finally, let 

00 

j=-oo 

We see that Q is the unit ball with infinitely many smooth dents which 
accumulate at (1,0) and (—1,0). Moreover, the automorphism group of Q is 
just the cyclic group {ipi} (see [LER], [GRK2] for the details). 

EXAMPLE 1.2 There is a bounded domain Q in C 2 with Lipschitz bound- 
ary with the property that S(Q) has just one point. 

To see this, let B be the unit ball in C 2 . For j G Z, let 



(p j {zi,z 2 ) 



2i-j Zl+ j/(j-2i) 2iz 2 /(j + 2i) 



2i + j 1 + Zl j/(j + 2%) ' 1 + Zl j/(j + 2%) 



The collection {ip^}JL_ OQ forms a cyclic subgroup of the automorphism group 
of B. One may see this by mapping B to the Siegel upper half plane U = 
{(wi,w 2 ) : Imwi > |u>2| 2 } and transferring the automorphims to U (see 
[KRA2] for the details of this process). In fact, on U, the automorphisms ip> 
become real translations in the w\ variable. 
Then 

lim ip 3 (z!,z 2 ) = (-1,0) , 

j— >±oo 

uniformly on compact subsets of (-21,-22) in B. 
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2 Examples Based on the Bidisc 



The examples in this section are certainly of some utility, but they are an- 
cillary to the main points we are trying to make in this paper. We include 
them for completeness and for interest's sake. 

EXAMPLE 2.1 There is a domain fifiC™, modeled on the bidisc D 2 , with 
the property that S(D) has dimension 0. 

EXAMPLE 2.2 There is a domain fiBC™, modeled on the bidisc D 2 , with 
the property that S(D) has dimension 1. 

EXAMPLE 2.3 There is a domain fiBC™, modeled on the bidisc D 2 , with 
the property that S(D) has dimension 2. 

EXAMPLE 2.4 There is a domain fifiC 1 , modeled on the bidisc D 2 , with 
the property that S(D) has dimension 3. 

Discussion of Examples 2.1-2.4: Let P G dD 2 be given by P = 
Let 

f f *i + 1/5 z 2 + 1/5 

^'^-U+a/s^'i+a/sK 

As usual, let ^ be the composition of y? with itself j times. Then 

lim <^'(P ) = (±1,±1). 

j— >-±oo 

If we set up a "dent" at Po, just as in Examples 1.1 and 1.2, and propagate 
the dent using the automorphisms (pi , then we produce a domain with auto- 
morphism group a cyclic group and two orbit accumulation points. This is 
a zero-dimensional S(Q). 

Now let us modify the discussion in the preceding paragraph by defining 
the disc automorphism 

and then considering the automorphisms 

^(21,22) = {^(zi),P 3 {z 2 )) , 

where A- 7 as usual denote the j-fold composition of A with itself and p denotes 
any nontrivial Mobius transformation £ \-t (( — a)/(l — a() of the disc D. 
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As usual, we construct a small dent at the boundary point (z, 0) and we 
propagate it using the automorphisms ipj,p for all integers j and all automor- 
phisms p of the unit disc. Then it is easy to see that the boundary orbit 
accumulation points of the resulting domain Q are points of the form 

{l}xdD and {-1} x dD . 

We see that, according to this construction, the dimension of S(Q) is 1. 
Now we modify the construction in the last two paragraphs by letting 

Vj(zi,z 2 ) = (X J (z 1 ),z 2 ) , 

We construct a domain with dents as usual — a modification of the bidisc 
D 2 — using these automorphisms. It is easy to see that, for the resulting Q, 
the set S(fl) is 

({-1}xD) U ({1}X£>). 

Thus S(D) is of dimension 2. 

Finally, if we consider just the usual bidisc D 2 with its full automorphism 
group, then the set S(D 2 ) is the entire topological boundary dD x DUD x dD. 
Thus S(D 2 ) has dimension 3. 

3 General Principles 

Now we wish to make some general assertions about the geometric nature of 
the orbit accumulation set 5(0). 

Proposition 3.1 Let QQC n be a smoothly bounded domain of finite type 
in the sense of Catlin/D'Angelo/Kohn (see [KRAI] for details). Then it is 
impossible for S(Q) to contain a relatively open subset U of dfl unless Q is 
biholomorphic to the unit ball B. 

Proof: This assertion is particularly easy to see in complex dimension 2. 
For if a relatively open subset U of the boundary has all points of geometric 
type greater than 2, then U is foliated by 1-dimensional complex analytic 
varieties (see [FRE])fl So every point of U is in fact of infinite type and 
that is a contradiction. Thus any relatively open subset U of dQ will contain 

4 One could at this point invoke the theorem of Kim [KIM] — at least in the convex 
case — to see that Q must be biholomorphic to the bidisc. That would be a contradiction. 
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points of type 2. Of course these points must be pseudoconvex by the result of 
[GRK1]. Hence the points are strongly pseudoconvex. But then the theorem 
of Bun Wong and Rosay (see [WON] and [ROS]) tells us that the domain Q 
is biholomorphic to the unit ball and hence S(Q) is the entire sphere. That 
proves the result. 

In C n with n > 1 we must instead use the structure theorem of Catlin 
[CAT] which tells us that the set of finite type points with the type greater 
than 2 forms a subset of the boundary of codimension at least 1. Hence any 
relatively open subset of the boundary will contain pseudoconvex points of 
type 2, and the argument is finished as before. □ 



Proposition 3.2 Let fi6C ra be a smoothly bounded domain of finite type 
in the sense of Catlin/ D'Angelo/Kohn (see [KRAI] for details). Then it is 
impossible for S(£l) to contain a subset E of <9f2 having positive (2n — 1)- 
dimensional Hausdorff measure unless Q is biholomorphic to the unit ball 
B. 

Proof: Reasoning as in the second part of the proof of the preceding propo- 
sition, we see that the set E must contain a strongly pseudoconvex point. It 
follows then from the Bun Wong/Rosay theorem that the domain must be 
biholomorphic to the unit ball. □ 



4 Closing Thoughts 

There is no Riemann mapping theorem in several complex variables. So 
we seek other means to compare and contrast domains up to biholomorphic 
equivalence. Certainly the automorphism group is one invariant that has 
proved to be both flexible and useful. 

One avenue that has been explored to a great extent is the Levi geometry 
of boundary orbit accumulation points (see [GRK3] and [ISK2]). The nature 
of the boundary orbit accumulation set is a much newer avenue of inquiry, 
and there remain many questions to be answered in this venue. We hope to 
explore some of them in future papers. 
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